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REMARKS 



Claims 3 1 and 32 have been amended to address the objection under 37 C.F.R. 
§ 1.75(c) and the rejection under 35 U.S.C. §112, second paragraph, and claims 27-29 have 
been canceled. Claims 18 and 33 have been amended to clarify that the objective function is 
a function of the monitored response performances following decisions and actions taken by 
the system. Claims 19-26, 30, and 34 have not been amended. Claim 28 has been rewritten 
as new claim 35. Upon entry of this amendment, claims 18-26 and 30-35 will be in the 
application. 

No new matter has been added by the proposed amendments. Also, no new issues 
have been raised as the amendments to claims 31 and 32 relate to bodily incorporating the 
subject matter of claim 18 into these formerly dependent claims. Also, new claim 35 raises 
no new issues as it corresponds to the subject matter of claim 28 except that it has been 
rewritten for clarity and to bodily incorporate the subject matter of claim 18. Entry of the 
proposed amendments after Final Rejection is requested. 
Objection to Claims 31-32 

The Examiner objected to claims 31 and 32 as allegedly being of improper dependent 
form for allegedly failing to limit claim 1 8 from which they depend. Claim 3 1 has been 
amended into independent form to incorporate the subject matter of claim 18 as suggested by 
the Examiner. Claim 32 has been amended to specify that the system of claim 3 1 may be a 
robot and the control apparatus controls the objective function of the robot so as to optimize 
the objective function of the robot. Withdrawal of the objection to claims 3 1 and 32 is 
requested. 

Rejection Under 35 U.S.C. §112, 2 nd Paragraph 

Claims 27-29 and 31-32 stand rejected under 35 U.S.C. 1 12, second paragraph, as 
allegedly being indefinite. Claims 27-29 have been canceled, thereby obviating this rejection 
with respect to those claims. Claims 3 1 and 32 have been amended to overcome this 
indefiniteness rejection. 

The Examiner finds claim 31 unclear as to whether the system in the language "a 
system having a control apparatus" is the same system as that recited in claim 18 or not. 
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Claim 31 has been amended to incorporate the limitations of independent claim 18 to clarify 
that only one system is being optimized in the claimed manner. 

The Examiner rejected claim 32 as indefinite as to how the robot specifically affects 
the elements of claims 1 8 and/or 3 1 . Claim 32 has been amended to recite that the system of 
claim 31 is a robot and that the control apparatus controls the objective function of the robot. 
The relationship of the subject matter of claim 32 to the subject matter of claim 3 1 is now 
believed to be clear. Claim 32 is thus believed to be definite as amended. 

Claim 28 has been rewritten as new claim 35 for purposes of clarification. Claim 35 
specifies that the system contains "subsystems" and that a candidate action of the system 
corresponds to the selection of a given subsystem and that the candidate actions are 
controlled in a fashion to optimize the performance of the entire system. Support for this 
language may be found, for example, in paragraphs [0135]-[0158] with respect to Figures 10- 
12 of the specification. Claim 35 also incorporates the language of claim 18 so as to specify 
how each subsystem is controlled. Claim 35 is thus believed to overcome the Examiner's 
indefiniteness concerns with respect to claim 28. Further examination of claim 35 is 
requested. 

The amendments to claims 31 and 32 are believed to overcome the Examiner's 
indefiniteness concerns with respect to claims 31 and 32, and new claim 35 is believed to 
overcome the Examiner's indefiniteness concerns with respect to claim 28. If the Examiner 
still has concerns about the definiteness of the claim language, she is encouraged to contact 
Applicant's undersigned representative to discuss claim changes to remove any such 
ambiguities. Withdrawal of the rejections of claims 27-29 and 31-32 under 35 U.S.C. 1 12, 
second paragraph, is solicited. 
Rejections Under 35 U.S.C. §103 

In the Official Action, claims 18-21, 24-25, 30-31, and 33-34 were finally rejected 

under 35 U.S.C. §103(a) as allegedly being obvious over Merriman et al. (US 2002/0099600) 

in view of Eppen et al. ("Quantitative Concepts for Management"). In addition, claims 22 

and 26 were finally rejected under 35 U.S.C. § 103(a) as allegedly being obvious over 

Merriman et al. and Eppen et al. further in view of McClave et al. ("A First Course in 

Business Statistics"); claim 23 was finally rejected under 35 U.S.C. § 103(a) as allegedly 

being obvious over Merriman et al. and Eppen et al. further in view of Jameson (US 
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6,032,123); and claims 27 and 32 were finally rejected under 35 U.S.C. § 103(a) as allegedly 
being obvious over Merriman et al. and Eppen et al. further in view of Strickland et al. (US 
5,790,407). These rejections are believed to be improper and are respectfully traversed. 

As noted by the Examiner, Merriman et al. disclose a system and method of 
controlling a system to optimize an objective function by selecting a candidate action, 
monitoring the system performance in response to the selected candidate action, storing a 
representation of the monitored response performance, choosing the next candidate action to 
optimize the objective function, and repeating the process. Thus, the Merriman et al. system 
makes decisions, stores the results and attempts to make improved decisions based on 
historical observations in a manner consistent with the prior art systems noted in the 
background portion of the specification. The Merriman et al. system thus has the same 
limitations as prior art "learning" systems discussed in the background portion of the 
specification. Moreover, as specifically acknowledged by the Examiner (page 17, lines 3-9, 
of the Final Rejection), Merriman et al. do not teach that the objective function is optimized 
by assessing the probability distributions of all the candidate actions in order to control the 
growth of regret, "where regret is a term that represents a system performance measure that 
considers the relative merit of exploration of one or more apparently non-best candidate 
actions with respect to the merit of exploring what appears to be the current best candidate 
action." For such a teaching of "regret," the Examiner refers to Eppen et al. However, as 
will be explained in detail below, the teachings of Eppen et al. do not control the growth of 
regret in the manner claimed either. Accordingly, the obviousness rejection must fail for lack 
of prima facie obviousness. 

Eppen et al describe their "Decision Framework" in connection with Figure 14.1 at 
page 503, second paragraph as follows: 

"In decision theory problems, the fundamental piece of data is a payoff table like 
Figure 14.1. In this figure alternative decisions are listed along the side of the table and the 
possible states of nature are listed across the top. The entries in the body of the table are the 
payoffs for all possible combinations of decisions and states of nature. 

The decision process proceeds as follows: 

1 . You, the decision maker, select one of the alternative decisions dl,. . . .dn. 
Suppose that you select dl . 
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2. After your decision is made, a state of nature occurs. Suppose state 2 occurs. 

3. The return you receive can now be determined from the payoff table." 
Eppen et al. thus teach processes (methods of controlling a system) to optimize 

objective functions (such as Minimax, Maximax, Maximin and Minimax Regret), where the 
objective function is a function of pay off performances. A single pay off performance is 
considered to arise when a particular decision interacts with a particular state of nature 
(scenario conditions). In the processes taught, Eppen et al. assume that given a particular 
decision and given a particular state of nature, that the pay off is a known quantity. These 
pay offs are stored in a predefined pay off table, such as in Figure 14.1. Importantly, these 
pay off values are not considered to be estimates based upon previous observations, and 
therefore the benefit of taking specific decisions to generate more exemplars and to improve 
the confidence in those estimated pay offs is not considered. 

The decision process of Eppen et al. contrasts with the claimed system where 
appraisals of estimated pay offs are dependent upon previous monitored response outcomes 
(given a particular decision and particular state of nature). This dependence on previous 
response outcomes is expressed directly in the claim language. For example, independent 
claim 18 recites: 

A system capable of making decisions (". . .the system being capable of 
performing a plurality of candidate actions. . ."); 

For which following a decision there is a response performance which is 
measurable (". . .and being capable of monitoring response performances of a performance of 
a respective candidate action. . ."); 

• Where that decision and response performance data is then stored, and where 

the stored decision and response performance data are therefore directly dependent upon the 
decisions taken, and where that stored data is used as the basis for appraisal of new decision 
scenarios ("storing according to the candidate action performed, a representation of said 
monitored response performance; . . .choosing which of a plurality of candidate actions is next 
performed so as to optimize said objective function by assessing, using the probability 
distribution of the response performances of all said plurality of candidate actions. . ."). 

In other words, the claimed system appraises the pay offs expected to arise from new 

decisions on the basis of historically observed pay offs, and where there exists uncertainties 
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in the estimates of these pay offs that are directly affected by the number of historical 
observations available upon which to base these pay off estimates. Eppen et al. do not handle 
such uncertainties in pay offs; on the contrary, as noted above, a predefined payoff table is 
provided. 

This difference is fundamental, as in the claimed system the decision process must 
consider the relative merit of further exploration of one or more apparently non-best 
candidate actions with respect to the relative merit of exploiting what appears to be the 
current best candidate action based on the finite set of historical response performances to 
date. The system must do this to manage the risk of being misled by estimated pay off 
outcomes which are based on few observations. The claimed system is therefore one in 
which the decision process requires the explicit control of "a system performance measure 
that considers the relative merit of exploration of one or more apparently non-best candidate 
actions, with respect to the relative merit of exploiting what appears to be the current best 
candidate action based on historical response performances to date" (claim 18). This 
requirement for balancing the exploration of apparent non-best performing decisions (actions) 
with the exploitation of the apparent best decisions (actions) is the central complexity and 
core subject matter of the claimed system. Eppen et al. always know with certainty the best 
candidate action for the given state of nature (because all the payoffs are known for each 
combination of decision and state of nature) and hence do not teach the minimization of the 
growth of regret under the condition where the true best candidate action is not known. 
Eppen et al. do not teach, and have no reason, to consider the relative merits of "exploration 
of one or more apparently non-best candidate actions, with respect to the relative merit of 
exploiting what appears to be the current best candidate action based on historical response 
performances to date" as claimed. Eppen et al. do not have to consider how to handle 
uncertainties in the payoff values of the payoff table, as they are considered to be known. 

As set forth in M.P.E.P. §§2142-2143.03, in order to establish a prima facie case of 

obviousness, patent examiners are required to establish three criteria: (1) there must be some 

suggestion or motivation, either in the references themselves or in the knowledge generally 

available to one of ordinary skill in the art, to modify the reference or to combine reference 

teachings; (2) there must be a reasonable expectation of success; and (3) the prior art 

reference, or combination of references, must teach or suggest all the claim limitations. The 
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examiner bears the initial burden of factually supporting any prima facie conclusion of 
obviousness. To make a proper obviousness determination, the examiner must "step 
backward in time and into the shoes worn by the hypothetical 'person of ordinary skill in the 
art' when the invention was unknown and just before it was made." In view of the available 
factual information, the examiner must make a determination as to whether the claimed 
invention "as a whole" would have been obvious at that time to a person of ordinary skill in 
the art. Importantly, a rejection based on these criteria must be based on what is taught in the 
prior art, not the applicant's disclosure. The applicant's disclosure may not be used as a 
blueprint from which to construct an obviousness rejection. 

In view of the fact that Merriman et al. and Eppen et al. taken together do not teach 
the claimed minimization in the growth of "regret," and hence do not teach all of the claimed 
features of the invention, even if the teachings of Merriman et al. could be combined with the 
teachings of Eppen et al. as the Examiner suggests, the invention of independent claim 18 
would not result. In particular, while the proposed combination could have taught one skilled 
in the art to minimize the growth of "regret" as taught by Eppen et al. based on known 
relationships stored in a payoff table, the proposed combination would not teach minimizing 
the growth of "regret" as claimed since the cited references do not teach or suggest 
"exploration" of "non-best candidate actions" as claimed in order to optimize the system 
performance even when the "payoff of a prospective candidate action is unknown (unlike 
Eppen et al. where the payoff is known). The Examiner has thus failed to establish prima 
facie obviousness and the rejection of claims 18 and 33 and all claims dependent thereon 
should be withdrawn. Similar features may also be found in independent claims 31 and 35; 
therefore, these claims are allowable for the same reasons. 

Moreover, the Examiner has further failed to provide a prima facie case of 

obviousness with respect to any claim since the Examiner has not met her burden of 

providing a suggestion or motivation, either in the references themselves or in the knowledge 

generally available to one of ordinary skill in the art, to combine the reference teachings. 

Instead, the Examiner has mistakenly stated that "it would have been obvious to one of 

ordinary skill in the art at the time of the invention to use probability distributions and the 

theory of regret in the iterative predictive model of Merriman et al. in order to increase the 

efficiency of utilizing advertising/action space by providing a decision framework with 
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which to analyze the various options." The general citations provided by the Examiner do 
not provide the requisite teachings, suggestions or motivations to combine the teachings of 
Merriman et al. and Eppen et al. in the manner contemplated by the Examiner. In any case, 
as noted above, the proposed combination clearly does not suggest the claimed system and 
method for minimizing the growth of regret in controlling a system. As a result, one skilled 
in the art would not be motivated to combine the teachings of Merriman et al. and Eppen et 
al. to provide a system and method for minimizing the growth of regret as claimed. 

In view of the above, if the Examiner elects to maintain the obviousness rejections of 
independent claims 18 and 33, the Examiner is again strongly urged to clearly articulate the 
evidence of suggestions, motivations, or knowledge possessed by those skilled in the art that 
would have led one skilled in the art to combine the teachings of the cited references to arrive 
at the claimed invention. In the absence of the requisite teachings and motivations to 
combine teachings to establish prima facie obviousness, the rejections of claims 18-34 as 
being obvious over Merriman et al. and Eppen et al. or any other cited prior art reference is 
improper and withdrawal of the obviousness rejections is respectfully solicited. 
Conclusion: 

For the above reasons, entry of the above amendment is appropriate after Final 
Rejection to place the present application in condition for allowance without introducing any 
new issues for the Examiner's consideration. A Notice of Allowability is respectfully 
solicited. 



Date: December 20, 2006 /Michael P. Dunnam/ 

Michael P. Dunnam 
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Facsimile: (215) 568-3439 
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